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Abstract 

The ability to evaluate and accurately articulate the outcomes associated with leadership 
development programs is critical to their continued financial and administrative support. With 
calls for outcome-based accountability, the need for rigorous evaluation is particularly relevant 
for those programs administered through the Cooperative Extension Sendee (CES). Using 
Bandura’s (1977) Social Learning Theory, a robust evaluation of agricultural leadership 
development programs administered through the CES in the southern region of the United States 
was conducted. Several key findings are examined, specifically: program participant demographics 
match parity requirements for CES programming: program participants are highly satisfied with 
their program experience; and program participants have held a large number of leadership roles 
within their communities and industries. 
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Introduction 

Since the mid-1960s agricultural leadership development programs (LDPs) have been 
providing opportunities for individuals involved in agriculture to cultivate their passion for 
leadership and the impact they can have in their industries and communities (Kellogg, 2000). 
Historically, these programs have focused on improving individuals’ capacities to serve as leaders 
within the agricultural and natural resource (ANR) industry (Whent & Leising, 1992). In particular 
ANR LDPs have encouraged individuals to act as opinion leaders within their networks of influence 
(Chiarelli, Stedman, Carter, & Telg, 2010; Lamm, Lamm, & Carter, 2014). 

Leadership development programming has been found to have numerous benefits for 
participants as well as the communities in which they live and work (Galloway, 1997). For 
example, an evaluation of a community LDP found participants had increased opportunities to 
network with other leaders and to identify projects to focus their collective efforts. The community 
benefitted from a large pool of concerned leaders that were able to focus their efforts towards 
making large positive changes within the community (Blair, 1988). 

Whent and Leising (1992) found that graduates of the California ANR LDP identified 
major benefits as “increased personal contacts and interaction with classmates, increased leadership 
skills...interaction with government and agricultural leaders and increased awareness and 
understanding of other societies and cultures” (p. 38). Additionally, a research study with the 
Florida ANR FDP found participants tended to act as opinion leaders regarding critical ANR issues 
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in their personal, professional, and community networks (Lamm et al., 2014). With programs in 
more than 28 states and over 7,200 alumni the potential benefit to individuals, the ANR industry, 
and the communities where individuals live is immense (Kellogg, 2000). 

Approximately half of all ANR LDPs evaluated by the W. K. Kellogg Foundation in 2000 
were operated out of the state land grant university system and were specifically associated with 
the Cooperative Extension Service (CES) administered through the university (Kellogg, 2000). 
More recently Kaufman, Rateau, Carter, and Strickland (2012) found that “most of the IAPAL 
[International Association of Programs for Agricultural Leadership] programs were housed within 
a university system, with more than half administered through their Land-Grant University’s 
Extension system” (p. 129). The pairing of ANR LDPs delivered through a CES organization has 
an “ability to deliver needed education to producers who need it” (Sparks, 2014, para. 5). However, 
as CES programs ANR LDPs also have a responsibility to objectively evaluate their programs and 
provide evidence of programmatic worth (Lamm & Israel, 2011; Morera et al., 2014). 

The Government Performance Results Act (GPRA) passed in 1993 and the Agricultural 
Research, Extension, and Education Reform Act (AREERA) passed in 1998 specifically address 
the legislative necessity for evaluation of CES programs. The GPRA targets how government tax 
dollars are being allocated with a focus on setting and measuring program goals, assessing 
performance, and holding funded programs accountable for results. These measures are intended 
to improve the tax paying public’s confidence in government funded programs, inform program 
managers decisions regarding program effectiveness, as well as serve as a direct input to 
congressional funding decisions (Ladewig, 1999). Performance based budgeting legislation for 
government-funded programs was further reinforced with the enactment of AREERA. The new 
legislations required plans of work, integrated reporting, and progress reporting for further 
governmental funding (Ladewig, 1999). 

The ability to evaluate and accurately articulate the outcomes associated with CES 
programs is critical to the continued financial and administrative support of those programs (Lamm, 
Israel, & Diehl, 2013). Additionally, evaluation data has been shown to ensure CES programs are 
relevant (Yang, Fetsch, McBride, & Benavente, 2009) and meeting the needs of their intended 
audience (McClure, Furhman, & Morgan, 2012). However, comprehensive evaluation of programs 
can be a challenge (Black & Earnest, 2009; Lamm et al., 2013). 

The results reported here represent an evaluation of ANR LDPs administered by the CES 
within the southern region of the United States. The evaluation was designed to address the need 
for a more robust measure of objective performance outcomes associated with CES programs 
(Lamm, Israel, & Harder, 2011) as well as ensuring that ANR LDPs are engaging an appropriate 
audience (Broun, Nilon, & Pierce II, 2009). Furthermore, this research is directly supportive of 
priority area six of the National Research Agenda: American Association for Agricultural 
Education 2011 - 2015 (Doerfert, 2011), vibrant, resilient communities. ANR LDPs administered 
through the CES are uniquely positioned to provide a pipeline of community leaders. Agricultural 
educators will benefit by having a robust benchmark of results upon which to evaluate future 
educational interventions intended to improve community vibrancy and resiliency. The 
establishment of robust benchmarks further support priority area five of the National Research 
Agenda, efficient and effective agricultural education programs (Doerfert, 2011), “in order to 
provide evidence of program effectiveness, agricultural education programs must collect and 
maintain accurate data that describes the quality and impact of its programs and outreach efforts at 
all levels” (p. 25). 


Conceptual Framework 

According to Morera et al. (2014) “Evaluation has evolved from being a necessity to being 
a priority in Extension” (p.73). Although legislation such as GRPA and AREERA may represent 
the necessity of CES program evaluation, they do not necessarily capture the priority (Lamm & 
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Israel, 2011). Effective evaluation has been shown to not only document program achievements, 
but also to contribute to a continuous improvement process where different aspects of the program 
are examined and adjusted as necessary (Frechtling, 2010; Patton, 2008). 

Kirkpatrick’s four-level evaluation model (1994) has been one of the most prominent 
evaluation models employed within learning environments; however, critics of the model indicate 
the model does not sufficiently account for contextual variables such as training participants 
themselves (Russ-Eft & Preskill, 2009). Theory-driven evaluation has been suggested as a means 
to improve the effectiveness, and utility, of program evaluation (Russ-Eft & Preskill, 2009). 
According to Smith (1994) theory-driven evaluation involves using a “program’s rationale or 
theory as the basis of an evaluation to understand the program’s development and impact” (p. 83). 
“Future learning, performance, and change evaluation models must include not only outcome 
variable but also the wide variety of variables affecting those outcomes” (Russ-Eft & Preskill, 2009, 
p. 97), consequently a synthesis of theory-driven and Kirkpatrick four-level models may be 
warranted. 

Social Learning Theory and the Kirkpatrick Four-Level Model 

According to Black and Earnest (2009), participants in ANR LDPs “undergo learning 
activities that form social relationships. The participant’s experiences occur through observation, 
modeling, cognition, and environment.. .these areas interact and lead to transformation within the 
individual, the organization, and the community” (p. 186). This process is consistent with 
Bandura’s (1977) social learning theory (SLT). As represented in Figure 1, SLT represents the 
reciprocal interaction between an individual, the environment, and the individual’s subsequent 
behavior (Bandura, 1977). For example, SLT would posit that an individual in a training, or 
leadership development, environment would be exposed to a program lead as well as other LDP 
participants. Individuals will learn not only from the program lead but also from each other. The 
collective experience of the lead, peers, and other environmental factors are layered into the broader 
context for LDP participants (Bandura, 1977). Based on these contextual observations, participant 
behavior would be expected to change; the modified behavior would then contribute back to the 
evolving context of the LDP by either being rewarded or punished; depending on the environmental 
consequence, the behavior may be reinforced or diminished (Bandura, 1977). After the LDP 
concludes, the participant behavior related to leadership-related tasks or roles would be expected 
to have changed accordingly (Bandura, 1977). 



Figure 1. Social Learning Theory (Bandura, 1977) 


For the more than 30 years SLT has been used as a theoretical underpinning to understand 
how and why individuals within organizations tend to function in similar yet unpredictable ways 
(Davis & Luthans, 1980). It is the cognitive process of the individual analyzing and making 
decisions based on the observations of the environment around them which leads to behavioral 
choices (Davis & Luthans, 1980). The process of observation and cognitive evaluation can also be 
understood as a form of ongoing vicarious learning (Manz & Sims, 1981). Bandura (1977) 
summarized the process by stating that, 
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By observing a model of the desired behavior, an individual forms an idea of how 
response components must be combined and sequenced to produce the new 
behavior. In other words, people guide their actions by prior notions rather than by 
relying on outcomes to tell them what they must do. (p. 35) 

In addition to organizational behavior (Davis & Luthans, 1980) and strategic management (Manz 
& Sims, 1981), SLT has also been found to be an appropriate model for educational settings 
(Crittenden, 2005). 

Individual 

Adults choose to participate in learning programs for a variety of reasons (Strong & Harder, 

2011) . Students in all settings bring a variety of cognitive presets to a learning environment; we 
expect differences in “goals, values, motives, attitude, personality, and ability” (Crittenden, 2005 
p. 961). However, consideration for the individual, and the role the participant plays within the 
learning context, has been absent within the existing Kirkpatrick four-level model approach (Russ- 
Eft & Preskill, 2009). However, a description of program participants has been identified as a 
meaningful evaluation measure within CES contexts, especially participant sex, race/ethnicity, and 
age (Guy, 2013). 

Research has shown demographic antecedents of cognitive processes can help agricultural 
educators predict, inform, and improve, program efficacy (Lamm, Carter, Stedman, & Lamm, 
2014). Agricultural educators can use demographic antecedents to make learning as personalized 
and effective as possible (Vincent & Ross, 2001). Additionally, these data may hold valuable 
programmatic insight as contextual variables (Russ-Eft & Preskill, 2009), and can help to ensure a 
diversity of perspectives are represented within a learning environment (Bandura, 1977). 

Environment 

ANR LDPs have been found to have similar structures with the average program being 21 
months long, including 12 seminars, and, on average, including 26 participants (Kaufman et al., 

2012) . Throughout the ANR LDP process, participants are exposed to a variety of social, political, 
and economic issues at the local state, national, and global levels (Lamm et al., 2014). Participants 
also have the opportunity to hear from, and interact with, emerging and established leaders across 
a number of industries and political positions. (Kaufman et al., 2012; Kellogg, 2000). 

Throughout the process the program director must ensure the appropriate experiences are 
planned and executed and that those activities are satisfying to participants (Crittenden, 2005). 
Within learning contexts participant satisfaction has been identified as a necessary pre-condition 
for maximized development (Kirkpatrick, 1994). Therefore, understanding levels of participant 
satisfaction has been shown to provide valuable programmatic environmental insights (Galindo- 
Gonzalez & Israel, 2010). Prom an integration perspective, level one of the Kirkpatrick four-level 
evaluation model gathers participant reaction to, or satisfaction with, a training event (Kirkpatrick, 
1994). 

Behavior 

Behavioral outcomes associated with ANR LDPs tend to be accrued in two categories: 
impact on the participant and impact on the community (Carter & Culbertson, 2012). Lor the 
individual, participation in ANR LDPs has been shown to provide educational opportunities that 
improve participant knowledge and skill levels across a number of areas (Carter & Culbertson, 
2012). Improved knowledge and skill has led to improved behavioral competence in a number of 
areas: critical thinking (Carter & Rudd, 2000), problem solving (Howell, Wir, & Cook, 1979), 
communication skills (Diem & Nikola, 2005), confidence (Carter & Rudd, 2000; Diem & Nikola, 
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2005; Howell et al., 1979), networking and team building skills (Earnest, 1996; Whent & Leising, 
1992), and knowledge of other cultures (Diem & Nikola, 2005). 

Previous research on participant behavior have tended to focus on self-reported data (Black 
& Earnest, 2009). According to Grove, Kibel, and Haas (2005) a more evidential measure of 
programmatic outcomes is an objective measure of the leadership roles participants have assumed. 
Level three of the Kirkpatrick four-level evaluation model specifically focuses on the degree to 
which participants apply what they learned following the training (Kirkpatrick, 1994). 
Consequently a quantitative measure of leadership positions may be an appropriate evaluative 
metric for Kirkpatrick level three within the behavior component of the programmatic theory, SLT. 

Conceptual Model 

Based on recommendations within the literature a theory based evaluation of ANR LDPs 
was proposed (Russ-Eft & Preskill, 2009; Smith, 1994). The integration of the underlying 
programmatic theory, SLT, and associated evaluation measures is visually represented in Figure 2. 
In particular, the individual factor of SLT is to be evaluated using participant characteristics as 
recommended by Guy (2013); the environment factor is to be evaluated by Kirkpatrick level one, 
program satisfaction; finally, the behavior factor is to be evaluated by Kirkpatrick level three, 
assuming leadership roles. 



Figure 2. Conceptual model 


Purpose and Objectives 

The purpose of this study was to evaluate ANR LDPs administered by the CES in the 
southern United States. The study sought to address the following research objectives: 

1. Describe the characteristics of individual ANR LDP participants. 

2. Describe the environmental perception among ANR LDP program participants through 
program satisfaction (Kirkpatrick level one). 

3. Describe behavior and programmatic application of ANR LDP program participants 
through self-reported leadership roles (Kirkpatrick level three). 
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Methods 

A descriptive research design was employed to address the research objectives. Data were 
collected through an online survey administered to a sample of ANR LDPs within the population 
of interest, specifically, alumni and current participants of ANR LDPs administered through the 
CES in the southern United States. 

Sample, Procedures, and Data Analysis 

A purposive sample was employed and included ANR LDPs that were active in the IAPAL 
organization and agreed to participate in the study. A purposive sample was deemed to be 
acceptable based on previous research indicating “purposive sampling has been useful in attitude 
and opinion surveys” (Ary, Jacobs, & Sorensen, 2010, p. 156), and “can produce reliable results 
since bias is contained even in severely heterogeneous populations” (Guarte & Barrios, 2006, p. 
284). 

The southern region was comprised of 15 states and territories identified by the United 
States Department of Agriculture (2014). These included: Alabama, Arkansas, Florida, Georgia, 
Kentucky, Louisiana, Mississippi, North Carolina, Oklahoma, South Carolina, Tennessee, Texas, 
Virginia, Puerto Rico, and the Virgin Islands. Of the 15, there were 8 states that had active ANR 
LDPs associated with the IAPAL organization and constituted the sample for the study: Arkansas, 
Florida, Georgia, Kentucky, Louisiana, Oklahoma, Texas, and Virginia. Amongst the remaining 
programs not included in the sample, one state had an ANR LDP but was not active in the IAPAL 
organization: North Carolina; two states no longer had active ANR LDPs: Alabama and South 
Carolina; two states were in the process of establishing ANR LDPs, but were not active: Tennessee 
and Mississippi; and two territories had no identifiable ANR LDPs: Puerto Rico and the Virgin 
Islands. 

Data were collected in the spring of 2014 using an online questionnaire developed in 
Qualtrics. Participants were contacted using Dillman, Smyth, and Christian’s (2008) tailored design 
method. Participants were contacted using a standard procedure. First, the program director emailed 
a pre-notice to all program participants approximately one week prior to the survey. Second, the 
researcher sent an email invitation to each participant that contained a link to the survey, a requested 
response date, and the Institutional Review Board verbiage notifying participants that there were 
no penalties or compensation for participating or not participating. Third, one week after the survey 
invitation the researcher sent a reminder email to non-responders. Fourth, one week after the first 
reminder the researcher sent a second reminder email to non-responders. Fifth, two days prior to 
the survey end date the researcher sent a third reminder email to non-responders. Sixth, the 
researcher sent a fourth and final email to non-responders on the survey end date. 

A total of 2060 questionnaire invitations were sent via email, with 286 returned based on 
inaccurate email addresses, for a net of 1774 potential respondents. A total of 960 questionnaires 
were completed for an overall response rate of 54%. Nonresponse analysis was conducted by 
comparing early and late respondents based on the recommendations of Lindner, Murphy, and 
Briers (2001). Participants that completed the survey prior to the first reminder message were 
considered early responders, whereas individuals that completed the survey after the third reminder 
message were considered late responders. No statistically significant differences between the two 
groups were observed. Consequently, non-response bias was not found to be an issue. Data were 
analyzed using the Statistical Package for the Social Sciences (SPSS) version 21. Descriptive 
statistics were calculated for all subsequent demographic, environment, and behavior variables 
(Ary et al., 2010). 
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Measures 

Individual. Within the SLT framework, individual attributes were measured using 
demographic data obtained through respondent self-report. Specifically, individuals were asked to 
report their sex, race/ethnicity, and age. For the purposes of the study respondent race and ethnicity 
were defined as self-perceived membership in population groups that define themselves by cultural 
heritage, language, physical appearance, behavior, or other characteristics (“Standards”, 1995, p. 
26). In this study, race was defined as: American Indian or Alaska native; Asian or Pacific Islander; 
Black or African American; White; or Other. Ethnicity was defined as either 
Hispanic/Latino(a)/Chicano(a) or not. These categories were based on United States of America 
Office of Management and Budget standards for the classification of Federal Data on Race and 
Ethnicity (“Standards”, 1995, p. 29). 

Regarding age, respondents were asked to indicate the month and year that they were bom. 
Based on the provided information, an approximate age at time of response was calculated. 
Additionally, respondents were asked to indicate the year they completed their ANR LDP. Based 
on the provided graduation year and year born an approximate age at graduation was calculated. 
The approximate age at graduation variable was created to provide additional programmatic 
insights (Rossi, Lipsey, & Freeman, 2004). 

Environment. Program environment was measured through participant satisfaction with 
the program (Galindo-Gonzalez & Israel, 2010; Kirkpatrick, 1994). Participants self-reported 
program satisfaction using a researcher-adapted scale developed by Judge, Boudreau, and Bretz 
(1994). The original measure was found to have a Cronbach’s a of .85. The scale was selected 
based on previous use within agricultural leadership research (Lamm, et al., 2014). The three-item 
scale assesses individual satisfaction (yes = 1, no = 0), how the individual typically felt about the 
program (1 = least satisfied to 5 = most satisfied), and finally percent of time satisfied (0% - 100%). 
The results from the three items were multiplied to calculate an overall construct score. Scores on 
the overall satisfaction construct ranged from zero to five. For example, an individual that indicated 
they were satisfied with the project team on the first question was coded as a one, if the individual 
then selected the most satisfied option the second question was coded as a five, in the final question 
if the individual indicated that they were satisfied with their program 85% of the time this was used 
as the final value. The index calculation would then be 1 x 5 x .85 or 4.25. 

Behavior. Evaluations of measurable participant behaviors are some of the most robust 
mechanisms through which the CES can quantify impact (Franz & Townson, 2008). In this study, 
behavior was quantified by summing the number of leadership roles participants reported holding. 
Individuals were asked to indicate if they had held any leadership roles since completing their ANR 
LDP. If an individual indicated yes, a follow up question asked for the names of up to five 
organizations they had held a leadership role within. For each organization that an individual 
identified, they were asked to identify the type of leadership roles they had held. Individuals could 
select from a list of nine leadership roles: President, President-elect, Vice President, Secretary, 
Treasurer, Board Member, Committee/Activity Chair, Volunteer, or Other. For example, if an 
individual indicated that they had served one organization, their local Farm Bureau, a follow up 
question collected information regarding roles held within that organization. If the individual had 
served as a board member and a volunteer in the organization, a total of two leadership roles would 
be recorded. There were no weightings assigned to leadership role type, a count of responses was 
determined to be sufficient based on the research objectives (Agresti & Finlay, 2009). 
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Results 

Individual Characteristics - Demographics 

The sample was 74.3% (n = 550) male and 25.7% (n = 190) female. Based on respondent 
provided birth month and year, approximate age at the time of the study was calculated. The average 
age of respondents at the time of the study was 49 (M = 48.84, SD = 10.63), with a range of ages 
between 24 and 80. The average age at graduation from their ANR LDP was 39 (M = 38.61, SD = 
8.37), with a range of ages between 22 and 65. 

Examining ethnicity, 1.7% (n = 16) of respondents identified themselves as 
Hispanic/Latino(a)/Chicano(a). In regards to respondents’ race, 92.2% (n = 676) identified 
themselves as White, 2.6% (n = 19) identified themselves as Black or African American, 1.6% (n 
= 12) identified themselves as American Indian or Alaska Native, one individual self-identified as 
Asian or Pacific Islander (0.1%), and 1.2% (n = 9) identified themselves as Other. A complete 
description of participant demographics can be found in Table 1. 

Table 1 


Demographics of Participants 


Characteristic 

n 

% 

Sex 

Male 

550 

74.3 

female 

190 

25.7 

Current Age 

Under 30 

20 

2.8 

30 to 39 

138 

19.3 

40 to 49 

198 

27.7 

50 to 59 

235 

32.8 

60 to 69 

116 

16.2 

70 and over 

9 

1.3 

Age at Graduation 

Under 30 

92 

13.4 

30 to 39 

316 

45.9 

40 to 49 

205 

29.8 

50 to 59 

65 

9.4 

60 to 69 

11 

1.6 

Race 

American Indian or Alaska Native 

12 

1.6 

Asian or Pacific Islander 

1 

0.1 

Black or African American 

19 

2.6 

White 

676 

92.2 

Other 

9 

1.2 

Hispanic Ethnicity 

16 

2.2 


Environment - Program Satisfaction 

The environmental perception among ANR LDP program participants was quantified vis- 
a-vis a measure of program satisfaction associated with level one of the Kirkpatrick model. Level 
of satisfaction with their ANR LDP, as reported by respondents, was calculated using the Judge et 


Journal of Agricultural Education 


128 


Volume 57, Issue 1, 2016 



Lamm et al. 


Evaluating Extension Based Leadership Development Programs 


al. (1994) scoring key. Of respondents, 95.6% (n = 710) were satisfied with their ANR LDP. 
Individuals’ feeling toward their ANR LDP had a minimum score of 1 and a maximum score of 5. 
Percentage of time satisfied ranged from a minimum of 2% to a maximum of 100% (M= 87.74%, 
SD = 13.40%). Program satisfaction scale scores had a possible range of zero to five. The ANR 
LDP satisfaction scale index had a minimum score of 0 and maximum score of 5.00 (M= 4.00, SD 
= 1.18). A complete description of participant response frequencies can be found in Table 2. 

Table 2 


Satisfaction with Program 


Item 

n 

% 

Satisfied Overall 

Yes 

710 

95.6 

No 

33 

4.4 

Feeling Toward ANR LDP 

Very Dissatisfied 

3 

0.4 

Dissatisfied 

11 

1.5 

Neither Satisfied Nor Dissatisfied 

31 

4.4 

Satisfied 

207 

29.1 

Very Satisfied 

460 

64.6 

% of Time Satisfied 

Under 25% 

4 

0.6 

25% to 49% 

14 

1.9 

50% to 74% 

66 

9.2 

75% to 99% 

538 

74.6 

100% 

99 

13.7 


Behavior - Assuming Leadership Roles 

To quantify behavior following participation in an ANR LDP, respondents were asked to 
indicate if they had served in leadership roles within professional organizations, their community, 
or as part of their personal life. Of the 633 respondents, 87% (n = 550) indicated they had 
participated in at least one leadership role. If a respondent indicated they had held a leadership role, 
a second question prompted them to indicate up to five organizations within which they had held 
such a role. A total of478 individuals documented one or more organizations. For each organization 
that an individual indicated as holding a leadership position within, a follow up question prompted 
them to indicate the type(s) of leadership position they held. A total of 439 individuals indicated 
they held at least one specific leadership role. Within each organization an individual could indicate 
up to nine different leadership roles. Total leadership roles were calculated by multiplying number 
of roles by frequency. Respondents had held a total of 2,778 leadership roles. Total leadership roles 
by type were calculated by multiplying role count by frequency. Respondents had served in board 
member roles the most ( n = 830), followed by presidencies (n = 467). Results are displayed in Table 
3. 
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Table 3 


Type and Number of Leadership Roles (n = 439) 


Type and number of roles 

/ 

% 

Board Member 

830 

29.9 

President 

467 

16.8 

Committee/Activity Chair 

425 

15.3 

Volunteer 

315 

11.3 

Vice President 

241 

8.7 

President-elect 

147 

5.3 

Secretary 

136 

4.9 

Treasurer 

119 

4.3 

Other 

98 

3.5 

Total 

2,778 

100.0 


Conclusions, Recommendations, and Implications 

Although there have been a number of studies evaluating individual ANR LDPs (e.g. Black 
& Earnest, 2009; Carter & Rudd, 2000; Whent & Leising, 1992) there has been a lack of studies 
focused on programmatic impacts of such programs within the broader context of CES. This is an 
understandable limitation given the complexity, accountability, and funding diversity within CES 
programs (Franz & Townson, 2008). Additionally, the observed uniqueness of program participants 
makes evaluation a challenge (Lamm et al., 2014). However, legislation such as GPRA and 
AREERA represent a mandate for all CES programs to track and report impacts and outcomes 
accurately (Lamm et al., 2013). Extending on previous research, which asked individuals to assess 
the capacity to act as a leader (e.g., Kelsey & Wall, 2003), this study asked for specific leadership 
roles held within organizations. Additionally, this study specifically addresses a need identified in 
previous agricultural education research to better understand agricultural and natural resource 
opinion leaders (Lamm et al., 2014) as well as supporting the National Research Agenda “in order 
to provide evidence of program effectiveness, agricultural education programs must collect and 
maintain accurate data that describes the quality and impact of its programs and outreach efforts at 
all levels” (Doerfert, 2011, p. 25). 

Individual 

One measure of CES program impact is tracking the demographics of participants for later 
reporting to the National Institute of Food and Agriculture (Guy, 2013). Ideally it would be possible 
to articulate participant characteristics from program records; however, such records are frequently 
incomplete or unavailable. In the absence of such records, survey data and analysis can provide 
valuable insights (Rossi et al., 2004). 

Based on the results of this study, the average participant was male (74%), white (92%), 
and in their late thirties at graduation (M = 38.61, SD = 8.37). Conducting a post hoc analysis 
relative to available USDA and Organization for Economic Co-operation and Development 
(OECD) data these results would indicate that ANR LDPs are maintaining necessary parity 
requirements for the targeted audience. Specifically, the OECD (2005) found that for every female 
employed in agriculture there were 30.5 males. Additionally, according to an analysis of the 2007 
Census for Agriculture Race, Ethnicity, and Gender Profiles in the southern region of the United 
States, approximately 92.2% of all farms are operated by individuals that identify themselves as 
White (USDA, 2009). From an age perspective, in 2007 the United States Department of 
Agriculture (USDA) found that the average age of a farm operator in the United States was 57, an 
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increase from 54 in 1997. The census data would indicate that principal farm operators are aging 
and that the trend could result in a non-sustainable farming workforce, therefore the need to prepare 
the next generation of younger operators is paramount (USDA, 2007). 

According to Bandura (1977), a diversity of individuals will ensure participants are 
exposed to a variety of viewpoints and can expand their social learning potential accordingly. These 
results also indicate that there tends to be a high degree of homophily, or participant similarity, 
within ANR LDPs. “Homophily limits people’s social worlds in a way that has powerful 
implications for the information they receive, the attitudes they form, and the interactions they 
experience” (McPherson, Smith-Lovin, & Cook, 2001, p. 415). Although previous research 
indicated there were costs associated with greater diversity, such as increased group conflict and 
lower cohesion (Staples & Zhao, 2006), there are also well-established benefits. For example, 
although diverse groups “can lead to conflict, they also provide an opportunity to solve problems 
in unique ways” (Robbins & Judge, 2009, p. 310). 

Audience parity is necessary but not sufficient to maximize the potential value associated 
with the individual aspect of SLT. “Homophily in race and ethnicity creates the strongest divides 
in our personal environments, with age, religion, education, occupation, and gender following in 
roughly that order” (McPherson et al., 2001, p. 415). ANR LDPs are encouraged to continue to 
recruit individuals from a variety of genders, races, and ages to ensure a variety of perspectives are 
represented. Current marketing and recruiting strategies may need to be analyzed to ensure an 
appropriately diverse audience is aware of opportunities for participation. Additionally, future 
research is suggested to specifically identify access barriers or facilitators across different 
audiences. 

Environment 

Based on SLT, environmental factors that represent the situation-consequence construct in 
which a learner is situated have a direct influence on behavioral outcomes (Davis & Luthans, 1980). 
In particular, environmental conditions support or suppress a learner’s ability to supply the 
necessary attention as well as retain the imparted material (Bandura, 1977). In this regard 
satisfaction was used as an environmental measure of learning environment (Crittenden, 2005). 
With over 93% of respondents indicating they were satisfied or very satisfied with their ANR LDP, 
program directors should be recognized for these outstanding results. As it relates to SLT, the high 
levels of environmental satisfaction should result in higher productivity, or desired behavioral 
outcomes (Robbins & Judge, 2009). 

From a CES program evaluation perspective, the assessment of program environment 
measured though participant satisfaction is well established within the CES literature (e.g., 
Galindo-Gonzalez & Israel, 2010). The results from this study indicated that participants have been 
very satisfied with their ANR LDP, and consequently the learning environment is meeting their 
needs. 

Agricultural educators can utilize these findings to further enhance programming efforts 
directed at professional audiences. The typical ANR LDP model of multiple in-person seminars 
where participants are expected to engage in the learning environment through experiential means 
provides empirical evidence for the efficacy of such teaching modalities. Furthermore, the results 
establish a benchmark of satisfaction that similar programs can use for comparison purposes. 

Behavior 

Based on recommendations within the literature, an evidential measure of programmatic 
outcomes was employed, specifically an objective measure of the leadership positions participants 
have assumed after completing their ANR LDP (Grove et al., 2005). In particular a total 2,778 
leadership roles were reported. Conservatively this would equate to 2.9 roles for each of the 960 
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respondents. Alternately, of the 960 respondents at least 46% had taken on a leadership position 
after completing their ANR LDP compared with 25.3% of United States citizens according to a 
Bureau of Labor Statistics report (2014). Although the 20% differential between groups is 
noteworthy, the lack of tests for statistical significance limits further inteipretation. 

Although a large number of leadership positions have been held, over half of respondents 
did not indicate taking on such a role following their participation in an ANR LDP. Based on this 
result more programmatic focus on assuming leadership positions is suggested. Additional research 
is recommended to follow up with those respondents that indicated that they have not taken on 
leadership positions in an attempt to identify barriers to doing so. Any programmatic barriers that 
are identified should be addressed through curriculum updates. Ongoing monitoring of intended 
behavioral outcomes amongst participants should provide one of the most robust evaluations of 
programmatic efficacy. Additionally, results indicated that participants are holding numerous 
leadership roles; however, what is unclear is how these results are impacted by participation in the 
ANR LDP. More rigorous longitudinal tracking of program alumni and requests for specific 
attribution of roles to programs is recommended. 

l imitations and Additional Recommendations 

Although the researchers employed a number of best practices in conducting this research, 
there are limitations that must be addressed. First, the response rate obtained for this study is 
acceptable given established guidelines (Baruch & Holtom, 2008); however, it is lower than ideal 
(Dillman et al., 2008). Secondly, the results were limited to those individuals that were provided to 
the researcher by program directors. Since all ANR LDPs are run independently, there is no central 
database against which to confirm provided information. 

Future research is recommended to extend beyond just those ANR LDPs that are 
administered through a state CES. A broader evaluation of ANR LDPs would provide greater 
visibility to the impact associated with such programs. Specifically, a review of programs 
administered through CES and non-CES systems, and from across all regions of the United States 
and internationally. Additionally, future research is suggested to investigate the antecedents of 
taking on leadership roles. Determining what the optimal conditions under which individuals take 
on leadership roles will help to inform agricultural educators develop the most appropriate and 
effective learning interventions possible. 

In summary, the results of this evaluation are generally positive; however, some areas of 
opportunity have been identified. Recruiting a more diverse set of participants and trying to 
encourage all participants to take on future leadership roles are particularly noteworthy. The 
establishment of benchmarks within a theory-based evaluation framework should inform future 
ANR LDP data analysis. Ideally more robust evaluations will lead to more impactful ANR LDPs 
and ultimately a more equipped participant base that can serve as leaders within their communities 
and the agricultural industry. 
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